
United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OK COMMERCE 
United States Patent and Trademark Office 
Address: COMMISSIONER FOR PATENTS 
P.O. Box 14S0 

Alexandria, Virginia 22313*1450 
www.uspto.gov 



APPLICATION NO. 



FILING DATE 



FIRST NAMED INVENTOR 



ATTORNEY DOCKET NO. CONFIRMATION NO. 



09/769,599 



28166 



01/25/2001 



Dina Kravets 



7590 08/11/2004 

MOSER, PATTERSON & SHERIDAN, LLP 
/SARNOFF CORPORATION 
595 SHREWSBURY AVENUE 
SUITE 100 

SHREWSBURY, NJ 07702 



SAR 13896 



4399 



EXAMINER 



BHATNAGAR, AN AND P 



ART UNIT 



PAPER NUMBER 



2623 

DATE MAILED: 08/1 1/2004 



Please find below and/or attached an Office communication concerning this application or proceeding. 



PTO-90C (Rev. 10/03) 



Office Action Summary 



Application No. 

09/769,599 



Examiner 

Anand Bhatnagar 



Applicant(s) 



KRAVETS ET AL. 



Art Unit 

2623 



-- The MAILING DA TE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 . 1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )□ Responsive to communication(s) filed on . 

2a)^ This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1 and 3-25 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) 13 Claim(s) 1U13 is/are allowed. 

6) E1 Claim(s) 7. 3-10. and 14-25 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)E3 The drawing(s) filed on 25 January 2001 is/are: a)^ accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
1 1 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12)D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)D All b)U Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 
2.Q Certified copies of the priority documents have been received in Application No. 



3.Q Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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Response to Arguments 

1. Applicant's amendment filed on 05/25/04 has been entered and made of 
record. 

2. Applicant has canceled claim 2. Applicant has amended claims 1 , 3, 10, 
and 1 1 . Currently, claims 1 and 3-25 are pending. 

3. Applicant in essence argues that the prior art of Vinod et al. ("Video Shot 
Analysis using Efficient Multiple Object Tracking", V.V. Vinod and Hiroshi 
Murase, NTT Basic Research Labs, 3-1 Morinosato-Wakamiya Atsugi-shi, 
Kanagawa, 243-01 Japan, IEEE, 1991) does not teach the step of determining a 
route comprising a trajectory of a first object having the trajectory of at least one 
other object. The prior art of Vinod et al. discloses to obtain motions of multiple 
objects between successive frames using histograms. Vinod et al. further 
teaches to see if two objects are close to each other in time by analyzing their 
respective trajectories (Vinod et al.; page 503 bottom paragraph in right column). 
One skilled in the art can determine from this collected data if any of the 
trajectories are the same or they are different for the number of moving objects in 
the successive frames. 

Another argument applicant makes is that the prior art of Cheng et al. 
("Querying Video Contents by Motion Example", Pu-Jien Cheng and Wei-Pang 
Yang, Department of Computer and Information Science, National Chiao Tung 
University, Hsinchin, Taiwn, R.O.C., IEEE, 2000) is not qualified as prior art since 
it was published in 05/2000 and this instant application has priority to a 
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provisional application (60/1 90,81 9) filed on 03/21,2000. Examiner disagrees. 
This reference of Cheng et al. does qualify as prior art, even though it was 
published in 05/2000, because it was presented in an International Symposium in 
1 1/99. Examiner welcomes applicant's representative to give the examiner a call 
if he would like the details of the symposium and a faxed copy will be sent. 
Examiner refers to the rejection below. 



DETAILED ACTION 
Claim Rejections - 35 USC § 102 

4 The following is a quotation of the appropriate paragraphs of 35 

U.S.C. 102 that form the basis for the rejections under this section made in this 
Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

Claims 1,3-8, 10, 14, 16, 17, and 19-21 are rejected under 35 
U.S.C. 102(b) as being anticipated by Vinod et al. ("Video Shot Analysis using 
Efficient Multiple Object Tracking", V.V. Vinod and Hiroshi Murase, NTT Basic 
Research Labs, 3-1 Morinosato-Wakamiya Atsugi-shi, Kanagawa, 243-01 Japan, 
IEEE, 1991). 

Regarding claim 1 : A method for performing motion analysis on a 
sequence of images, where said sequence of images captures a plurality of 
objects each moving along a trajectory in an imaged area, said method 
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comprising (page 501 under section #2 1 st paragraph, where motion analysis is 
performed wherein multiple motions are determined of multiple objects from 
successive video frames): 

extracting motion information for each of said plurality of objects contained 
in said sequence of images (page 504 section 3.2, where the motion of multiple 
objects is determined and their respective trajectories determined); and 

determining spatial patterns from said extracted motion information (page 
504 section 3.2, where a trajectory map is determined of the multiple trajectories 
that are determined. The trajectory map is read as the spatial patterns.). 

determining a route comprising a trajectory of a first object having the 
same trajectory of at least one other object. 

Vinod et al. discloses to obtain motion parameters for multiple moving 
objects within a video sequence. Vinod et al. further discloses to obtain a 
trajectory of each moving object and to get the start and ending positions of each 
object's trajectory. Vinod et al. does not teach to obtain a trajectory of a first 
object having the same trajectory of at least one other object. Vinod discloses to 
obtain the trajectories of each object's motion and their respective start and 
ending positions and to generate a trajectory map from this data (section 3.2). 
Inherently, one skilled in the art can analyze this trajectory map obtain 
information such as how many objects are moving, which ones start and/or end 
at the same positions, which ones have the same/different trajectories, etc. 



Application/Control Number: 09/769,599 Page 5 

Art Unit: 2623 

Regarding claim 3: It is rejected for the same reason as claim 2 and for 
the following limitations of: 

determining whether said trajectory of a second object is within a 
threshold distance said trajectory of said first object (section 3.1 2 nd paragraph); 
and 

including, if said trajectory of said second object is within the threshold 
distance, said trajectory of said second object in said route (section 3.1 2 nd 
paragraph). 

Regarding claim 4: The method wherein said determining of spatial 
patterns comprises: 

determining a source point "start" and a destination point "end" from said 
trajectory of said plurality of objects (section 3.2, where the start "source" and 
end "destination" positions of the trajectories are determined). 

Regarding claim 5: It is rejected for the same reason as claim 2 and for 
the following limitations of: 

determining whether a number of trajectories originating from a location is 
greater than a threshold number; and 

identifying, if the number of trajectories originating from the location is 
greater than the threshold number, the location as said source point. 

Vinod et al. discoses to determine a map of all the trajectories obtain of all 
the moving objects in a video sequence. Vinod further analyzes for the start and 
end positions of each trajectory and to use thresholding to acquiring objects that 
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are within a close proximity to each other. Vinod et al. does not teach to have a 
threshold set for a number of trajectories starting from a certain location and to 
set this as a source point if the number of trajectories starting at the same point is 
above the threshold. Inherently one skilled in the art can analyze a trajectory map 
to see how many start at the same place or end up at the same place since the 
start and end positions of each trajectory is determined and to label them as 
preferred, such as source point, beginning point, start point, etc. Thresholding is 
a well known concept in image processing and examiner takes Official Notice. 

Regarding claim 6: It is rejected for the same reason as claim 2 and for 
the following limitations of: 

determining whether a number of trajectories ending at a location is 
greater than a threshold number; and 

identifying, if the number of trajectories ending at the location is greater 
than the threshold number, the location as said destination point. 

Vinod et al. discoses to determine a map of all the trajectories obtain of all 
the moving objects in a video sequence. Vinod further analyzes for the start and 
end positions of each trajectory and to use thresholding to acquiring objects that 
are within a close proximity to each other. Vinod et al. does not teach to have a 
threshold set for a number of trajectories ending at a certain location and to set 
this as a destination point if the number of trajectories ending at the same point is 
above the threshold. Inherently one skilled in the art can analyze a trajectory map 
to see how many start at the same place or end up at the same place since the 
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start and end positions of each trajectory is determined and to label them as 
preferred, such as end point, finish point, destination point, etc. Thresholding is a 
well known concept in image processing and examiner takes Official Notice. 

Regarding claim 7: It is rejected for the same reason as claim 2 above and 
for the following limitations of: wherein said source point and said destination 
point are determined using a clustering process. Examiner takes Official Notice 
because the method of clustering is well known in image processing such as for 
locating objects in an image(s). 

Regarding claim 8: The method further comprising: 

determining spatio-temporal patterns from said determined spatial 
patterns along a time dimension (section 3.1 last paragraph and section 3.2, 
where the trajectories "spatial patterns" are determined in time and a trajectory 
map "spatio-temporal patterns" produced. 

Regarding claim 10: wherein said determining of spatio-temporal patterns 
comprises: determining a periodicity of at least one trajectory in said route. 

It is rejected for the same reasons as claims 2 and 8 combined. 

Regarding claim 14: 

determining whether said trajectory in said first route is time correlated 
with said trajectory in said second route. 

Regarding claim 16: It is rejected for the same reason as claim 1 above. 
It is rejected for the same reasons as claims 2 and 8 combined. 
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Regarding claim 17: It is inherent that the video signal needs to transmit 
the signal from a video camera to an apparatus which analyzes data in the video 
images, such as motion since a video camera usually does not analyze images 
for motion. 

Regarding claims 19 and 20: It is rejected for reasons of the combination 
of claims 2 and 16. 

Regarding claim 21 : It is rejected for the same reasons in claim 8. 



Claim Rejections - 35 USC § 103 

5 The following is a quotation of 35 U.S.C. 103(a) which forms the basis for 

all obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

A.) Claim 9 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Vinod et al. ("Video Shot Analysis using Efficient Multiple Object Tracking", V.V. 
Vinod and Hiroshi Murase, NTT Basic Research Labs, 3-1 Morinosato-Wakamiya 
Atsugi-shi, Kanagawa, 243-01 Japan, IEEE, 1991) and Weil et al. (U.S. patent 
6,177,885 B1). 

Regarding claim 9: The method wherein said determining of 
spatio-temporal patterns comprises: determining a busy time for said route, 
where the busy time represents a time when a number of trajectories for said 
plurality of objects along said route is greater than a threshold number. 
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Vinod et al. discloses to obtain motion parameters for multiple moving 
objects within a video sequence. Vinod et al. does not teach to determine the 
busy time for a route. Weil et al. teaches to obtain the busy time or day for a 
route of moving objects (read as "traffic flow") (Weil et al. col. 3 lines 2-5, 12-15, 
and 22-30, and col. 4 lines 55-60). It would have been obvious to one skilled in 
the art to combine the teaching of Weil et al. to that of Vinod et al. because they 
are analogous in monitoring the movement of moving objects using a video 
sequence. One in the art would have been motivated to incorporate the teaching 
of Weil et al. to that of Vinod et al. in order to detect the occurrence of accidents 
(Weil et al.; col. 2 lines 40-44). 

B.) Claims 15, 18, and 22-25 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Vinod et al. ("Video Shot Analysis using Efficient Multiple 
Object Tracking", V.V. Vinod and Hiroshi Murase, NTT Basic Research Labs, 3-1 
Morinosato-Wakamiya Atsugi-shi, Kanagawa, 243-01 Japan, IEEE, 1991) and 
Cheng et al. ("Querying Video Contents by Motion Example", Pu-Jien Cheng and 
Wei-Pang Yang, Department of Computer and Information Science, National 
Chiao Tung University, Hsinchin, Taiwn, R.O.C., IEEE, 2000). 

Regarding claims 15,18, and 22: They are rejected for the same reasons 
as claim 2 above. As for the following limitations of performing a query from a 
plurality of patterns stored on a database and for displaying the results. 
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Vinod et al. discloses to obtain trajectories of all the moving objects in a 
video sequence. Vinod et al. does not teach to perform a query of these 
trajectories which are stored in a database and to display the results. Cheng et 
al. teaches to perform an image query of trajectorie(s), of objects in motion, in a 
video sequence stored in a database (Cheng et al.; page 287 abstract, page 288 
section 2.1 , section 3, section 4.2, and section 4.3). It would have been obvious 
to one skilled in the art to combine the teaching of Cheng et al. to that of Vinod et 
al. because they are analogous in observing trajectories of moving objects in a 
video sequence. One in the art would have been motivated to incorporate the 
teaching of Cheng et al. to the system of Vinod et al. in order to obtain data about 
a specific object(s) in motion, such as it's trajectory, speed, angle of motion, etc. 

Regarding claims 23 and 24: They are rejected for the same reasons as 
claim 2 above. 

Regarding claim 25: It is rejected for the same reasons as claim 8 above. 

Allowable Subject Matter 

6 Claims 11-13 are allowed. 

Conclusion 

7 THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of 
time policy as set forth in 37 CFR 1.136(a). 
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A shortened statutory period for reply to this final action is set to expire 
THREE MONTHS from the mailing date of this action. In the event a first reply is 
filed within TWO MONTHS of the mailing date of this final action and the advisory 
action is not mailed until after the end of the THREE-MONTH shortened statutory 
period, then the shortened statutory period will expire on the date the advisory 
action is mailed, and any extension fee pursuant to 37 CFR 1 .136(a) will be 
calculated from the mailing date of the advisory action. In no event, however, will 
the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 



Bhatnagar whose telephone number is 703-306-5914, whose supervisor is 
Amelia Au whose number is 703-308-6604, group receptionist is 703-305-4700, 
and group fax is 703-872-9306. 
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